System and method for sensorized user interface

ABSTRACT

A system and method for is provided for operation of an orthopedic system. The system includes a load sensor for converting an applied pressure associated with a force load on an anatomical joint, and an ultrasonic device for creating a low-power short-range ultrasonic sensing field within proximity of the load sensing unit for assessing alignment. The system can adjust a strength and range of the ultrasonic sensing field according to position. It can report audible and visual information associated with the force load and alignment. Other embodiments are disclosed.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a Continuation-In-Part of U.S. patent applicationSer. No. 11/844,329 filed on Aug. 23, 2007 claiming the priority benefitof U.S. Provisional Patent Application No. 60/839,742 filed on Aug. 24,2006, the entire contents of which are hereby incorporated by reference.

BACKGROUND

The present embodiments of the invention generally relate to the fieldof electronic accessory devices, and more particularly to sensors andinput pointing devices. Medical systems and other sensing technologiesare generally coupled to a display. Interaction with the display canoccur via mouse, keyboard or touch screen. There are times when audiblefeedback is advantageous.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A is an illustration of a graphical user interface of a system andearpiece to provide audio feedback in accordance with one embodiment;

FIG. 1B is a block diagram of the earpiece in accordance with oneembodiment;

FIG. 2 is an illustration a receiver and wand component of thenavigation system in accordance with one embodiment;

FIG. 3 is an exemplary method for audio feedback of load balance andalignment with the navigation system in accordance with one embodiment;

FIG. 4 is an illustration of a load sensing unit placed in a joint ofthe muscular-skeletal system for measuring a load parameter inaccordance with one embodiment;

FIG. 5A is an illustration of an orthopedic navigation system forassessing cutting jig orientation and reporting anatomical alignment inaccordance with one embodiment;

FIG. 5B illustrates an integrated alignment and balance Graphical UserInterface (GUI) in accordance with one embodiment;

FIG. 6 illustrates a load sensing unit for measuring a parameter of themuscular-skeletal system in accordance with an example embodiment;

FIG. 7 illustrates one or more remote systems in an operating room inaccordance with an example of FIG. 8 to illustrate the detectordetecting one of visual, motion, or audio queue from a user inaccordance with an example embodiment;

FIG. 8 illustrates the detector detecting one of visual, motion, oraudio queue from a user in accordance with an example embodiment;

FIG. 9 illustrates a robotic tool coupled to the remote system inaccordance with an example embodiment; and

FIG. 10 illustrates a sensor array of detectors in accordance with anexample embodiment.

DETAILED DESCRIPTION

While the specification concludes with claims defining the features ofthe invention that are regarded as novel, it is believed that theinvention will be better understood from a consideration of thefollowing description in conjunction with the drawing figures, in whichlike reference numerals are carried forward.

Broadly stated, a system and method is provided having both userinterface and orthopedic parameter measurement capability. In oneembodiment, the system provides quantitative measurements ofmuscular-skeletal system and insertion of prosthetic components. Anavigation portion of the system provides user interface control via oneor more wand pointing devices and a receiver device. The wand can beused to identify points of interest in three-dimensional space and foruser input. The system can incorporate user feedback in the context of asurgical work-flow. The wand can also be affixed to an object to trackits movement and orientation within proximity of the receiver. Ameasurement portion of the system includes sensors for measuring aparameter of the muscular-skeletal system. An example of parametermeasurement is a force, pressure, alignment or load measurement. In oneembodiment, the sensors reside in a prosthetic component of anartificial joint system. The prosthetic component when installedprovides quantitative measurement data that can be used to assess theknee joint. Sensory feedback and guidance provides audio and visualindication of work flow steps and the wand's location and orientation.

Referring to FIG. 1A, an exemplary medical system 100 for measuring,assessing, and reporting quantitative data is shown. In a non-limitingexample, system 100 can measure parameters of the muscular-skeletalsystem while simultaneously providing position and alignmentinformation. The medical system 100 includes a graphical user interface(GUI) 108, a load sensing unit 170 and a navigation device 180wirelessly coupled thereto. The load sensing unit 170 provides a measureof applied force and can be used to measure load balance. In oneembodiment, load sensing unit 170 is housed in a prosthetic componentfor trial measurement. The surgical navigation device 180 projects anultrasonic sensing field 181 and adjusts its energy and range based on aproximity of its component devices, and with respect to the load sensingunit 170. It measures and reports an alignment that corresponds to theload magnitude and load balance measurements reported by the loadsensing unit 170.

The load sensing unit 170 converts a force applied thereto within ananatomical joint into an electric signal, for example, by way of polymersensing, piezoelectric transduction, or ultrasonic waveguidedisplacement to measure a load balance of the force. The load andalignment indicators presented in the GUI 108 provide quantitativefeedback of the measured parameters during a work flow. The system 100also includes an earpiece 190 that audibly presents an indicator of thealignment and the load balance responsive to completion of one or morework flow steps of the surgical procedure. The work flow steps are, forexample, associated with the indentifying and tracking three-dimensional(3D) points in the ultrasonic sensing field 181, for instance, ananatomical location or instrument orientation. One example of anavigation system and user interface for directing a control action isdisclosed in U.S. patent application Ser. No. 12/900,878, the entirecontents of which are hereby incorporated by reference.

A remote system 104 serves as a display that is wirelessly coupled tothe surgical navigation device 180 and graphically presents informationrelated to the load balance and alignment via the GUI 108. Remote system104 can be placed outside of the sterile field of an operating room butwithin viewing range of the medical staff. In one embodiment, remotesystem is a laptop computer. Alternatively, remote system 104 can be anapplication specific equipment that comprises a display and a processor,including but not limited to, a mobile device. Remote system 104 candisplay the work flow with measurement data for assessment. Remotesystem 104 can include a processor to analyze the quantitativemeasurement data to provide feedback during a procedure that can bevisual, audible, or haptic.

Referring to FIG. 1B, a block diagram of the earpiece 190 is shown. Theearpiece 190 provides audio feedback of parameters reported by the loadsensing unit 170, the navigation device 180 and the GUI 108. Theearpiece 190 can include a transceiver 191 that can support singly or incombination any number of wireless access technologies including withoutlimitation Bluetooth, Wireless Fidelity (WiFi), ZigBee and/or othershort or long range radio frequency communication protocols. Thetransceiver 191 can also provide support for dynamic downloadingover-the-air to the earpiece 190. Next generation access technologiescan also be applied to the device. The processor 192 can utilizecomputing technologies such as a microprocessor, Application SpecificIntegrated Chip (ASIC), and/or digital signal processor (DSP) withassociated storage memory 193 such a Flash, ROM, RAM, SRAM, DRAM orother like technologies for controlling operations of the earpiecedevice 190. The processor 192 can also include a clock to record a timestamp. The speaker 195 can couple to the processor 192 through an analogto digital converter or a general processor input/output (GPIO) pin forproviding acoustic feedback, for example, audio beeps or other audibleindications. The power supply 194 can utilize common power managementtechnologies such as replaceable batteries, supply regulationtechnologies, and charging system technologies for supplying energy tothe components of the earpiece 190 and to facilitate portableapplications. The remote system 104 can also provide power for operationor recharging via USB or direct coupling of the earpiece 190.

Referring to FIG. 2, exemplary components of the navigation device 180are shown, specifically, a wand 200 and a receiver 220 for creating thethree-dimensional ultrasonic sensing field. The wand 200 is a hand-helddevice with a size dimension of approximately 8-10 cm in width, 2 cmdepth, and an extendable length from 10-12 cm. The receiver 220 has sizedimensions of approximately 1-2 cm width, 1-2 cm depth, and a length of4 cm to 6 cm. Neither device is however limited to these dimensions andcan be altered to support various functions (e.g, hand-held, coupled toobject). The current size permits ultrasonic tracking of the wand tipwith millimeter spatial accuracy up to approximately 2 m in distance.Not all the components shown are required; fewer components can be useddepending on required functionality, for instance, whether the wand isused for isolated point registration, continuous wand tracking withoutuser input or local illumination, or as integrated devices (e.g., laptopdisplay).

The wand 200 includes sensors 201-203 and a wand tip 207. The sensorscan be ultrasonic transducers, Micro Electro Mechanical Element (MEMS)microphones, electromagnets, optical elements (e.g., infrared, laser),metallic objects or other transducers for converting or conveying aphysical movement to an electric signal such as a voltage or current.They may be active elements in that they are self powered to transmitsignals, or passive elements in that they are reflective or exhibitdetectable magnetic properties. The wand 200 is used to register pointsof interest (see points A, B, C), for example, along a contour of anobject or surface, which can be presented in a user interface (seeremote system 104 FIG. 1A). As will be discussed ahead, the wand 200 andreceiver 220 can communicate via ultrasonic, infrared andelectromagnetic sensing to determine their relative location andorientation to one another. Other embodiments incorporatingaccelerometers to provide further positional information is discussed inU.S. patent application Ser. No. 12/982,944 filed Dec. 31, 2010 theentire contents of which are incorporated by reference.

In one embodiment, the wand 200 comprises three ultrasonic transmitters201-203 each can transmit ultrasonic signals through the air, anelectronic circuit (or controller) 214 for generating driver signals tothe three ultrasonic transmitters 201-203 for generating the ultrasonicsignals, a user interface 218 (e.g., button) that receives user inputfor performing short range positional measurement and alignmentdetermination, a communications port 216 for relaying the user input andreceiving timing information to control the electronic circuit 214, anda battery 215 for powering the electronic circuitry of wand 200. Thewand 200 may contain more or less than the number of components shown;certain component functionalities may be shared as integrated devices.

The wand tip 207 identifies points of interest on a structure, forexample, an assembly, object, instrument or jig in three-dimensionalspace, though is not so limited. The wand tip 207 does not requiresensors since its spatial location in three-dimensional space isestablished by the three ultrasonic transmitters 201-203 arranged at thecross ends. However, a sensor element can be integrated on the tip 207to provide ultrasound capabilities (e.g., structure boundaries, depth,etc.) or contact based sensing. In such case, the tip 207 can be touchsensitive to registers points responsive to a physical action, forexample, touching the tip to an anatomical or structural location. Thetip can comprise a mechanical or actuated spring assembly for suchpurpose. In another arrangement it includes a capacitive touch tip orelectrostatic assembly for registering touch. The wand tip 207 caninclude interchangeable, detachable or multi-headed stylus tips forpermitting the wand tip to identify anatomical features while thetransmitters 201-203 remain in line-of-sight with the receiver 220 (seeFIG. 1A). These stylus tips may be right angled, curved, or otherwisecontoured in fashion of a pick to point to difficult to touch locations.This permits the wand to be held in the hand to identify via the tip207, points of interest such as (anatomical) features on the structure,bone or jig. One such example of an ultrasonic navigation system isdisclosed and derived from U.S. Pat. No. 7,725,288 and application Ser.No. 12/764,072 filed Apr. 20, 2010 the entire contents of which arehereby incorporated by reference.

The user interface 218 can include one or more buttons to permithandheld operation and use (e.g., on/off/reset button) and illuminationelements to provide visual feedback. In one arrangement, a 5-statenavigation press button 209 can communicate directives to furthercontrol or complement the user interface. It can be ergonomicallylocated on a side of the wand to permit single handed use. The wand 200may further include a haptic module with the user interface 218. As anexample, the haptic module may change (increase/decrease) vibration tosignal improper or proper operation. The wand 200 includes materialcoverings for the transmitters 201-202 that are transparent to sound(e.g., ultrasound) and light (e.g., infrared) yet impervious tobiological material such as water, blood or tissue. In one arrangement,a clear plastic membrane (or mesh) is stretched taught; it can vibrateunder resonance with a transmitted frequency. The battery 215 can becharged via wireless energy charging (e.g., magnetic induction coils andsuper capacitors).

The wand 200 can include a base attachment mechanism 205 for coupling toa structure, object or a jig. As one example, the mechanism can be amagnetic assembly with a fixed insert (e.g., square post head) to permittemporary detachment. As another example, it can be a magnetic ball andjoint socket with latched increments. As yet another example, it can bea screw post or pin to a screw. Other embodiments may permit sliding,translation, rotation, angling and lock-in attachment and release, andcoupling to standard jigs by way of existing notches, ridges or holes.

The wand 200 can further include an amplifier 213 and the accelerometer217. The amplifier enhances the signal to noise ratio of transmitted orreceived signals. The accelerometer 217 identifies 3 and 6 axis tiltduring motion and while stationary. The communications module 216 mayinclude components (e.g., synchronous clocks, radio frequency ‘RF’pulses, infrared ‘IR’ pulses, optical/acoustic pulse) for signaling tothe receiver 220 (FIG. 2B). The controller 214, can include a counter, aclock, or other analog or digital logic for controlling transmit andreceive synchronization and sequencing of the sensor signals,accelerometer information, and other component data or status. Thebattery 215 powers the respective circuit logic and components. Theinfrared transmitter 209 pulses an infrared timing signal that can besynchronized with the transmitting of the ultrasonic signals (to thereceiver).

Additional ultrasonic sensors can be included to provide anover-determined system for three-dimensional sensing. The ultrasonicsensors can be MEMS microphones, receivers, ultrasonic transmitters orcombination thereof. As one example, each ultrasonic transducer canperform separate transmit and receive functions. One such example of anultrasonic sensor is disclosed in U.S. Pat. Nos. 7,414,705 and 7,724,355the entire contents of which are hereby incorporated by reference. Theultrasonic sensors can transmit pulse shaped waveforms in accordancewith physical characteristics of a customized transducer forconstructing and shaping waveforms.

The controller 214 can utilize computing technologies such as amicroprocessor (uP) and/or digital signal processor (DSP) withassociated storage memory 208 such a Flash, ROM, RAM, SRAM, DRAM orother like technologies for controlling operations of the aforementionedcomponents of the device. The instructions may also reside, completelyor at least partially, within other memory, and/or a processor duringexecution thereof by another processor or computer system. AnInput/Output port permits portable exchange of information or data forexample by way of Universal Serial Bus (USB). The electronic circuitryof the controller can comprise one or more Application SpecificIntegrated Circuit (ASIC) chips or Field Programmable Gate Arrays(FPGAs), for example, specific to a core signal processing algorithm.The controller can be an embedded platform running one or more modulesof an operating system (OS). In one arrangement, the storage memory maystore one or more sets of instructions (e.g., software) embodying anyone or more of the methodologies or functions described herein.

The receiver 220 comprises a processor 233 for generating timinginformation, registering a pointing location of the wand 200 responsiveto the user input, and determining short range positional measurementand alignment from three or more pointing locations of the wand 200 withrespect to the receiver 220. It includes a communications interface 235for transmitting the timing information to the wand 200 that in responsetransmits the first, second and third ultrasonic signals. The ultrasonicsignals can be pulse shaped signals generated from a combination ofamplitude modulation, frequency modulation, and phase modulation. Threemicrophones 221-223 each receive the first, second and third pulseshaped signals transmitted through the air. The receiver 220 shape canbe configured from lineal as shown, or in more compact arrangements,such as a triangle shape. The receiver 220 can also include anattachment mechanism 240 for coupling to bone or a jig. As one example,the mechanism 240 can be a magnetic assembly with a fixed insert (e.g.,square post head) to permit temporary detachment. As another example, itcan be a magnetic ball and joint socket with latched increments.

The receiver 220 can further include an amplifier 232, thecommunications module 235, an accelerometer 236, and processor 233. Theprocessor 233 can host software program modules such as a pulse shaper,a phase detector, a signal compressor, and other digital signalprocessor code utilities and packages. The amplifier 232 enhances thesignal to noise of transmitted or received signals. The processor 233can include a controller, counter, a clock, and other analog or digitallogic for controlling transmit and receive synchronization andsequencing of the sensor signals, accelerometer information, and othercomponent data or status. The accelerometer 236 identifies axial tilt(e.g., 3/6 axis) during motion and while stationary. The battery 234powers the respective circuit logic and components. The receiverincludes a photo diode 241 for detecting the infrared signal andestablishing a transmit time of the ultrasonic signals to permitwireless infrared communication with the wand. The receiver 200 maycontain more or less than the number of components shown; certaincomponent functionalities may be shared or therein integrated.

The communications module 235 can include components (e.g., synchronousclocks, radio frequency ‘RF’ pulses, infrared ‘IR’ pulses,optical/acoustic pulse) for local signaling (to wand 200). It can alsoinclude network and data components (e.g., Bluetooth, ZigBee, Wi-Fi,GPSK, FSK, USB, RS232, IR, etc.) for wireless communications with aremote device (e.g., laptop, computer, etc.). Although externalcommunication via the network and data components is hereincontemplated, it should be noted that the receiver 220 can include auser interface 237 to permit standalone operation. As one example, itcan include 3 LED lights comprising an indicator 224 to show three ormore wand tip pointing location alignment status. The user interface 237may also include a touch screen or other interface display with its ownGUI for reporting positional information and alignment.

The processor 233 can utilize computing technologies such as amicroprocessor (uP) and/or digital signal processor (DSP) withassociated storage memory 108 such a Flash, ROM, RAM, SRAM, DRAM orother like technologies for controlling operations of the aforementionedcomponents of the terminal device. The instructions may also reside,completely or at least partially, within other memory, and/or aprocessor during execution thereof by another processor or computersystem. An Input/Output port permits portable exchange of information ordata for example by way of Universal Serial Bus (USB). The electroniccircuitry of the controller can comprise one or more ApplicationSpecific Integrated Circuit (ASIC) chips or Field Programmable GateArrays (FPGAs), for example, specific to a core signal processingalgorithm or control logic. The processor can be an embedded platformrunning one or more modules of an operating system (OS). In onearrangement, the storage memory 238 may store one or more sets ofinstructions (e.g., software) embodying any one or more of themethodologies or functions described herein. A timer can be clockedwithin the processor 233 or externally.

The wireless communication interface (Input/Output) 239 wirelesslyconveys the positional information and the short range alignment of thethree or more pointing locations to a remote system. The remote systemcan be a computer, laptop or mobile device that displays the positionalinformation and alignment information in real-time as described ahead.The battery 234 powers the processor 233 and associated electronics onthe receiver 220. The sensor 228 can measure air flow (or speed) of theair flowing through the device in a defined time segment. It can also becoupled to a tube (not shown) for more accurately measuring acousticimpedence. The thermistor 229 measures ambient air temperature infractional units and can report an analog value within a tuned range(e.g., 30 degree span) that can be digitized by the processor 233. Oneexample of a device for three-dimensional sensing is disclosed in U.S.patent application Ser. No. 11/683,410 entitled “Method and Device forThree-Dimensional Sensing” filed Mar. 7, 2007 the entire contents ofwhich are hereby incorporated by reference.

In a first arrangement, the receiver 220 is wired via a tetheredelectrical connection (e.g., wire) to the wand 200. That is, thecommunications port of the wand 200 is physically wired to thecommunications interface of the receiver 220 for receiving timinginformation. The timing information from the receiver 220 tells the wand200 when to transmit and includes optional parameters that can beapplied to pulse shaping. The processor 233 on the receiver 220 employsthis timing information to establish Time of Flight measurements in thecase of ultrasonic signaling with respect to a reference time base.

In a second arrangement, the receiver 220 is communicatively coupled tothe wand 200 via a wireless signaling connection. An infraredtransmitter 209 on the wand 200 transmits an infrared timing signal witheach transmitted pulse shaped signal. Wand 200 pulses an infrared timingsignal that is synchronized with the transmitting of the ultrasonicsignals to the receiver 220. The receiver 220 can coordinate theemitting and the capturing of the plurality of ultrasonic pulses viawireless infrared synchronized communication from the wand 200. Thereceiver 220 can include a photo diode 241 for determining when theinfrared timing signal is received. In this case the communications portof the wand 200 is wirelessly coupled to the communications interface ofthe receiver 220 by way of the infrared transmitter and the photo diode241 for relaying the timing information to within microsecond accuracy(˜1 mm resolution). The processor 233 on the receiver 220 employs thisinfrared timing information to establish the first, second and thirdTime of Flight measurements with respect to a reference transmit time.

Referring to FIG. 3, a method 300 for navigated system operationproviding audio feedback is shown. The method 300 can be practiced withmore or less than the number of steps shown. To describe the method 300,reference will be made to FIGS. 1-2, 4 and 5 although it is understoodthat the method 300 can be implemented in any other suitable device orsystem using other suitable components. Moreover, the method 300 is notlimited to the order in which the steps are listed. In addition, themethod 300 can contain a greater or a fewer number of steps than thoseshown in FIG. 3.

The method 300 can start at step 311 after the surgical work flow hascommenced and the navigation system has been calibrated and configured.The remote system 104 visually presents the GUI and provides work-flowvisual guidance and audio content during the surgical procedure. Oneexample of a workflow user interface is disclosed in U.S. patentapplication Ser. No. 12/901,094 entitled “Orthopedic Navigation Systemwith Sensorized Devices” filed Oct. 8, 2010 the entire contents of whichare hereby incorporated by reference. The referenced disclosure aboveprovides reference work steps to a total knee replacement surgery thatcan be used as example to the method 300 herein.

At step 312, an applied anatomical force within an anatomical joint,such as the knee, can be converted into an electric signal, for example,by way of piezoelectric transduction to measure a load balance of theapplied anatomical force in the joint. The conversion can be performedand reported by the load sensing unit 170 previously shown in FIG. 1A.The load sensing unit is an integrated wireless sensing modulecomprising an i) encapsulating structure that supports sensors andcontacting surfaces and ii) an electronic assemblage that integrates apower supply, sensing elements, transducers, biasing spring or springsor other form of elastic members, an accelerometer, antennas andelectronic circuitry that processes measurement data as well as controlsall operations of energy conversion, propagation, detection and wirelesscommunications. One exemplary method of wireless parameter sensing andreporting is disclosed in U.S. patent application Ser. No. 12/825,724filed Jun. 29, 2010 the entire contents of which are hereby incorporatedby reference. In yet other arrangements, the load sensing unit 170 caninclude piezoelectric, capacitive, optical or temperature sensors ortransducers to measure the compression or displacement. It is notlimited to ultrasonic transducers and waveguides. One exemplary methodof force sensing is disclosed in U.S. patent application Ser. No.12/826,329 filed Jun. 29, 2010 the entire contents of which are herebyincorporated by reference.

At a step 314, an energy and range of a projected ultrasonic sensingfield is adjusted based on a proximity measurement. In general, thenavigation system can be used to measure a mechanical axis of themuscular-skeletal system. In the example, the mechanical axiscorresponds to the position of the femur and tibia in relation to oneanother. The mechanical axis is used for determining bone cut angles forreceiving prosthetic components. Human anatomy can vary substantiallyover a large population. For example, the energy can be increasedthereby extending the range of the ultrasonic sensing field formeasuring position and alignment of a tall patient. Conversely, theenergy can be decreased when devices are positioned in close proximityto prevent overloading transducers with a large signal.

At a step 316, a report is generated from quantitative measurementsrelated to alignment and load. Measurements on load magnitude, loadbalance, and alignment are included. The report can indicate loadmagnitude and load balance over a range of motion for the alignment. Theremote system provides visualization of the alignment and loadmeasurements to support the prosthetic component installation. Moreover,the remote system can process the quantitative data to project outcomesbased on the information.

At a step 318, an audible signal can be sent to the ear piece thatpresents the load balance and alignment. The audible presentation canprovide feedback to the surgeon during one or more steps of the workflow. Along with provided visualization, the audible signal can aid orsupport the translation of quantitative data to the subjective feel ortactile feedback present when the surgeon works on and moves theprosthetic components. At step 321, the method 300 can end.

Briefly referring now to FIG. 4, an illustration 400 of the load sensingunit 170 is shown for assessing and reporting load forces in ananatomical joint in accordance with an exemplary embodiment. Theillustration shows the device 170 measuring a force, pressure, or loadapplied by the muscular-skeletal system, more specifically, the kneejoint. It can report a load on each component and a balance of a medialcondyle forces and a lateral condyle forces on the tibial plateau. Theload is generally naturally balanced to provide even wear of the tibialsurface, or a tibail prosthetic tray when included with a kneereplacement surgery. The level and location of wear may vary based on aperson's health and bone strength; hence, the need for the detection offorce, location and balance. The load balance also contributes to theanatomical alignment (e.g., mechanical axis) and how this alignment ismaintained.

In the illustration, device 170 collects load data for real-time viewingof the load forces over various applied loads and angles of flexion(when the knee is bent) or extension (when the knee is straight). As oneexample, it can convert an applied pressure associated with a force loadon the anatomical joint into an electric signal via mechanical sensor,polymer sensor, strain gauge, piezoelectric transduction, ultrasonicwaveguide displacement, or other transduction means. The sensing insertdevice 170 measures the level and distribution of load at various pointson the prosthetic components (412/422) and transmits the measured loaddata by way data communication to the remote system 104 (or receiverstation) for permitting visualization. This can aid the surgeon inproviding soft tissue release (where ligaments may be titrated) or otheradjustments needed to achieve optimal joint balancing.

Load sensing unit 170 has a contacting surface that couples to themuscular-skeletal system. As shown, a first and a second contactingsurface respectively couple to a femoral prosthetic component 412 and atibial prosthetic tray component 422. Device 170 is designed to be usedin the normal flow of an orthopedic surgical procedure without specialprocedures, equipment, or components. One example of a load sensor unitis disclosed U.S. patent application Ser. No. 12/825,638 entitled“System and Method for Orthopedic Load Sensing Insert Device” filed Jun.29, 2010 the entire contents of which are hereby incorporated byreference. Typically, one or more natural components of themuscular-skeletal system are replaced when joint functionalitysubstantially reduces a patient quality of life. A joint replacement isa common procedure in later life because it is prone to wear over time,can be damaged during physical activity, or by accident.

A joint of the muscular-skeletal system provides movement of bones inrelation to one another that can comprise angular and rotational motion.The joint can be subjected to loading and torque throughout the range ofmotion. The joint typically comprises two bones that move in relation toone another with a low friction flexible connective tissue such ascartilage between the bones. The joint also generates a naturallubricant that works in conjunction with the cartilage to aid in ease ofmovement. Load sensing unit 170 mimics the natural structure between thebones of the joint. Load sensing unit 170 has one or more articularsurfaces on which a bone (femur 410) or a prosthetic component (412) canmoveably couple. A knee joint is disclosed for illustrative purposes butload sensing unit 170 is applicable to other joints of themuscular-skeletal system. For example, the hip, spine, and shoulder havesimilar structures comprising two or more bones that move in relation toone another. In general, load sensing unit 170 can be used between twoor more bones (or prosthetic components) allowing movement of the bonesduring measurement or maintaining the bones in a fixed position.

Returning back to FIG. 3 and further supporting step 314, an energy andrange of a projected ultrasonic sensing field can be adjusted based onproximity of the components of the navigation device, and also at times,proximity and position of the load sensing unit 170. In the later, thenavigation device 180 identifies the location of the load sensing unit170 and ensures it is properly positioned for assessing alignment. Thispermits for controlled measurement of an alignment corresponding with aload balance of the applied anatomical force. It calculates therelationship of the forces on each compartment (e.g., condyles) with thevarus and valgus alignment conditions. The adjusting is performed by thenavigation device 180 previously shown in FIG. 1 and can adjust anultrasonic waveform transmit sensitivity in accordance with a movementand proximity of the wand 200 to the receiver 220, for example, tomatch, compensate or characterize transmitter characteristics with thesound field environment (e.g., noise, airflow, temperature, etc.).

As an example, the receiver 220 signals the wand 200 to transmitultrasonic waves at a higher energy level and repetition rate as afunction of the distance there between. This can improve averaging andcontinuous tracking performance. It can also modify the transmitted waveshape depending on the distance or as a function of the detectedenvironmental parameters. The shaping can occur at the processor on thereceiver 220 (or pre-stored as a waveform in memory) responsive to anindex flag, and the digital waveform is then communicated to the wand200 which then transmits the pulse in accordance with the wave shape.Alternatively, the wand 200 can include a local memory to store waveshapes, and the receiver 220 transmits an index to identify which storedwaveshape the wand 200 should transmit. One means of mapping athree-dimensional sensory field with the navigation device 180 isdisclosed in U.S. Pat. No. 7,788,607 and patent application Ser. No.12/853,987 entitled “Method and System for Mapping Virtual Coordinates”filed Dec. 1, 2006 the entire contents of which are hereby incorporatedby reference.

The wand 200 can provide additional user interface control via a softkey 218. The receiver 220 can precisely track the wand 200 up to 2 mdistances and report its position (among other parameters) on the GUI108. It can track multiple wand locations if more than one wand ispresent, for example, to report positional information or movement ofmultiple points or objects. Although the receiver 220 is shown asstationary and the wand 200 as free to move, the reverse is true; thereceiver 220 can track its own movement relative to a stationary wand200 or a plurality of wands. U.S. Pat. No. 7,834,850 and patentapplication Ser. No. 12/900,662 entitled “Navigation Device ProvidingSensory Feedback” filed Oct. 8, 2010 disclose principles of operationemployed herein; the entire contents of which are hereby incorporated byreference. The receiver can calculate an orientation of the wand at thewand tip of the wand (4 cm or more away from any one of the plurality ofultrasonic transmitters) with respect to a virtual coordinate system ofthe receiver; and wirelessly transmit the alignment information withrespect to three or more registered points in the ultrasonic sensingfield in the virtual coordinate system of the receiver via a wirelessradio frequency communication link.

FIG. 5A depicts an exemplary embodiment of the navigation system 500 foruse as an alignment tool in total knee replacement procedures. Thenavigation system 500 includes the receiver 220, a mounting wand 230 andthe wand 200; components of the navigation device 180 from FIG. 1. Thewand 200 can be held in the hand for registering points and also becoupled to a guide or jig 520 as shown. This permits for navigating theguide or jig in view of the registered points. The guide jig supportsthe cutting of bones in relation to the mechanical axis of the joint formounting prosthetic components thereto. It also includes the remotesystem 104 (e.g., laptop, mobile device, etc.) for presenting thegraphical user interface (GUI) 108. The GUI 108 allows the user tovisualize a navigated workflow and can be customized to the orthopedicprocedure. The navigation system 500 as illustrated also includes theload sensing unit 170, shown inserted between the femur and tibia of theknee joint, and the earpiece 190 which receives from the remote system104 audible messages related to the work flow GUI 108.

The system 500 assesses and reports in real-time the position of thesepoints, or other registered points, by way of the GUI 108 on the remotesystem 108. It provides visual and auditory feedback related to cuttingjig orientation and alignment, such as audible acknowledgements, hapticsensation (e.g., vibration, temperature), and graphical feedback (e.g.,color, measurements, numbers, etc). Another example of an alignment toolused in conjunction with the embodiments herein contemplated isdisclosed in U.S. Patent Application 61/498,647 entitled “OrthopedicCheck and Balance System” filed Jun. 20, 2011 the entire contents ofwhich are hereby incorporated by reference.

Returning back to FIG. 3 and further supporting step 316, the alignmentand the load balance is reported at specific work flow times responsiveto one or more steps during the work flow, for example, those associatedwith registering three-dimensional points in the ultrasonic sensingfield. The steps associated therewith can include requiring the surgeonto identify an anatomical landmark, for example, by pressing the wandbutton, tracing out a contour of a bone or identifying points on a guideor jig, or navigating a guide into a predetermined orientation(position, location, angle, etc.). Other steps can include femur headidentification, the use of a different cutting jig (femur, tibia, 4in1block rotation, etc.) requiring detachment and reattachment of the wand200, controlling the user interface 108 (e.g., pagination, markers,entries, etc.), assessing/performing bone cuts, assessing/performingsoft tissue release, insertion/repositioning/removal of the load sensingunit 170 or navigation device 180 components or other commonly practicedwork flow steps. One exemplary embodiment of providing sensory feedbackin a navigated workflow with the sensorized tools is disclosed in U.S.patent application Ser. No. 12/900,878 filed Oct. 8, 2010 entitled“Navigation System and User Interface For Directing a Control Action”,the entire contents of which are hereby incorporated by reference.

Briefly referring again to FIG. 5, the receiver 220 precisely tracksboth wands 200-230 and reports their position on the GUI 108 as part ofthe navigated workflow procedure. During the procedure, the receiver 220is rigidly affixed to the femur bone for establishing a base (orvirtual) coordinate system. The mounting wand 230 is rigidly affixed tothe tibia for establishing a second coordinate system with respect tothe base coordinate system. The wand 200 is used to register points ofinterest with the receiver 220. The points of interest can be on a boneor on cutting jigs 520 used during surgery. Thereafter, the wand 200 canbe rigidly coupled to the cutting jig 520 (femur/tibia) to establishvirtual cut angles for making corresponding cuts on the bones. Thenavigation system 500 reports real-time alignment of the cutting jigs520 and bones by way of direct communication between the wands 220-230and the receiver 220; no computer workstation is required there between.

In one arrangement, the wand emits a plurality of ultrasonic pulses froma plurality of ultrasonic transmitters configured to transmit theultrasonic pulses. The receiver with a plurality of microphones capturesthe ultrasonic pulses and digitally samples and stores a history ofreceived ultrasonic waveforms in memory. It estimates from the storedultrasonic waveforms a time of flight between transmitting and receivingof the ultrasonic pulses, and identifies a location of the wand from thetime of flight measurements received at the plurality of microphones. Itthen calculates for the plurality of ultrasonic waveforms stored in thememory a phase differential between the ultrasonic waveforms andpreviously received ultrasonic waveforms, and updates the location ofthe wand from a mathematical weighting of the time of flight with thephase differential.

In addition to visual and auditory feedback provided by the GUI 108 onthe laptop 104, the load balance and alignment information can beaudibly presented via the earpiece 190 responsive to completion of theone or more steps during the work flow as recited in step 318 in thedescription of method 300 of FIG. 3.

The earpiece 190 may receive such audio content directly from the laptop104 or the receiver 220, which is also equipped with wirelesscommunication (e.g, Bluetooth). The receiver 220 and laptop 104 canprovide Bluetooth profiles such as Human Interface Device Profile (HID),Headset Profile (HSP), Hands-Free Profile (HFP) or Serial Port Profile(SPP) for low complexity overhead, though is not limited to these. TheBluetooth profile provides a wireless interface specification forBluetooth-based communication between devices.

As another example, audio feedback can be provided for any of thesesteps or other steps associated with assessing both balance andalignment, for instance, audibly indicating the force level, forinstance, by increasing a media loudness (e.g. tone, beep, syntheticmessage) or frequency corresponding to the level while evaluatingalignment. As another example, the earpiece 190 can provide syntheticvoice messages for a “to do” step in the work flow of the GUI 108. Itcan also be configured to chime or respond to a predetermined parameter,for example, when ideal balance and alignment has been achieved.

This information can also be visually reported via the GUI 108previously shown in the navigation system 500 of FIG. 5A. That is, thealignment and balance information be numerically shown with othergraphical features (e.g., color, size, etc.) for emphasis. One benefitof audible feedback is that it permits the surgeon to look and listenseparately to alignment and balance. For instance, the GUI can providevisual alignment information while the earpiece provides audio level forforce. In situations where two earpieces are used, the force balance maybe applied as pan balance between earpieces. Balance can be noted whenthe sound parameters (e.g., loudness, tone, etc.) from the left andright earpiece are equal. The visual and auditory combination can assistthe surgeon in comprehending load balance and alignment during surgery.

FIG. 5B illustrates an integrated visual information GUI 590 forefficiently displaying balance in view of alignment, for instance, toshow the mechanical axis 591 and the load lin 592 overlay with referenceto anatomical load forces. It provides visual feedback and audibleinformation over the earpiece. Briefly, the Receiver 220 and Wand (TX)230 render of the mechanical axis 591; that is, the alignment betweenbone coordinate systems. The load sensor 170 data provides the GUIrendering of the load line 592; that is, the location and loading of theforces on the knee compartments that contribute to the overall stabilityof the prosthetic knee components. The overlay GUI 590 can also simulatesoft tissue anatomical stresses associated with the alignment andbalance information. For example, the GUI 590 can adjust a size andcolor of a graphical ligament object corresponding to a soft tissueligaments according to the reported alignment and balance information.The ligament object 594 can be emphasized red in size to show excesstension or stress as one example. Alternatively, ligament object 595 canbe displayed neutral green if the knee compartment forces for alignmentand balance result within an expected, or acceptable, range, as anotherexample. Such predictive measurements of stress based on alignment andbalance data can be obtained, or predetermined, from widespread clinicalstudies or from measurements made previously on the patient's knee, forexample, during a clinical, or pre-op.

FIG. 6 illustrates the load sensing unit 170 for measuring a parameterof the muscular-skeletal system in accordance with an exampleembodiment. In general, and with respect to FIG. 4, it is the insertcomponent 170 between the femur prosthetic component 412 and the tibiaprosthetic component 422 which together comprises a joint replacementsystem to allow articulation. In the example, load sensing unit 170 is aprosthetic insert that is a wear component of a knee joint replacementsystem. The prosthetic insert has one or more articular surfaces(602/604) that allow joint articulation. As shown, load sensing unit 170has two articular surfaces 602 and 604 with underlying sensors formeasuring load. The articular surface is low friction and can absorbloading that occurs naturally based on situation or position. Thecontact area between surfaces of the articulating joint can vary overthe range of motion. The articular surface of a permanent insert willwear over time due to friction produced by the prosthetic componentsurface contacting the articular surface during movement of the joint.Ligaments, muscle, and tendon hold the joint together and motivate thejoint throughout the range of motion.

Load sensing unit 170 is an active device having a power source,electronic circuitry, transmit capability, and sensors within the bodyof the prosthetic component. As disclosed herein, load sensing unit 170can be used intra-operatively to measure parameters of themuscular-skeletal system to aid in the installation of one or moreprosthetic components. As shown, operation of load sensing unit 170 isshown as a knee insert to illustrate operation and measurement of aparameter such as loading and balance. Load sensing unit 170 can beadapted for use in other prosthetic joints having articular surfacessuch as the hip, spine, shoulder, ankle, and others. Alternatively, loadsensing unit 170 can be a permanent active device that can be used totake parameter measurements over the life of the implant. One example ofthe wireless load sensor unit is disclosed U.S. patent application Ser.No. 12/825,724 entitled “Wireless Sensing Module for Sensing a Parameterof the Muscular-Skeletal System” filed Jun. 29, 2010 the entire contentsof which are hereby incorporated by reference.

In both a trial, or permanent insert, load sensing unit 170 issubstantially equal in dimensions to a passive final prosthetic insert.In general, the substantially equal dimensions correspond to size andshape that allow the insert to fit substantially equal to the passivefinal prosthetic insert. In the intra-operative example, the measuredloading and balance using load sensing unit 170 as a trial insert wouldbe substantially equal to the loading and balance seen by the finalinsert. It should be noted that load sensing unit 170 forintra-operative measurement could be dissimilar in shape or have missingfeatures that do not benefit the trial during operation. The insert ispositionally stable throughout the range of motion equal to that of thefinal insert. The exterior structure of load sensing unit 170 is formedfrom at least two components.

In the embodiment shown, load sensing unit 170 comprises a supportstructure 600 and a support structure 608. Support structures 600 and608 have major support surfaces that are loaded by the muscular-skeletalsystem. As previously mentioned, load sensing unit 170 is shown as aknee insert to illustrate general concepts and is not limited to thisconfiguration. Support structure 600 has an articular surface 602 and anarticular surface 604. Condyles of a femoral prosthetic componentarticulate with surfaces 602 and 604. Loading on the prosthetic kneejoint is distributed over a contact area of the articular surfaces 602and 604. In general, accelerated wear occurs if the contact area isinsufficient to support the load. A region 606 of the support structure600 is unloaded or is lightly loaded over the range of motion. Region606 is between the articular surfaces 602 and 604. It should be notedthat there is an optimal area of contact on the articular surfaces tominimize wear while maintaining joint performance. The contact locationcan vary depending on the position within the range of motion of themuscular-skeletal system. Problems may occur if the contact area fallsoutside a predetermined area range within articular surfaces 602 and 604over the range of motion.

In one embodiment, the location where the load is applied on articularsurfaces 602 and 604 can be determined by the sensing system. This isbeneficial because the surgeon now has quantitative information wherethe loading is applied. The surgeon can then make adjustments that movethe location of the applied load within the predetermined area usingreal-time feedback from the sensing system to track the result of eachcorrection. The support structure 608 includes sensors and electroniccircuitry 612 to measure loading on each articular surface of theinsert. A load plate 616 underlies articular surface 602. Similarly, aload plate 618 underlies articular surface 604. Force, pressure, or loadsensors (not shown) underlie load plates 616 and 618. In one embodiment,load plates 616 and 618 distribute the load to a plurality of sensorsfor determining a location where the load is applied. In the example,three sensors such as a piezo-resistive sensor underlies a correspondingload plate. The sensors are located at each vertex of the triangularshaped load plate.

Although the surface of load plates 616 and 618 as illustrated, areplanar they can be conformal to the shape of an articular surface. Aforce, pressure, or load applied to articular surfaces 602 and 604 isrespectively coupled to plates 616 and 618. Electronic circuitry 612 isoperatively coupled to the sensors underlying load plates 616 and 618.Plates 616 and 618 distribute and couple a force, pressure, or loadapplied to the articular surface to the underlying sensors. The sensorsoutput signals corresponding to the force, pressure, or load applied tothe articular surfaces, which are received and translated by electroniccircuitry 612. The measurement data can be processed and transmitted toa receiver external to load sensing unit 170 for display and analysis.In one embodiment, the physical location of electronic circuitry 612 islocated between articular surfaces 602 and 604, which correspond toregion 606 of support structure 600. A cavity for housing the electroniccircuitry 112 underlies region 606. Support structure 608 has a surfacewithin the cavity having retaining features extending therefrom tolocate and retain electronic circuitry 612 within the cavity. Theretaining features are disclosed in more detail hereinbelow. Thislocation is an unloaded or a lightly loaded region of the insert therebyreducing a potential of damaging the electronic circuitry 612 due to acompressive force during surgery or as the joint is used by the patient.In one embodiment, a temporary power source such as a battery,capacitor, inductor, or other storage medium is located within theinsert to power the sensors and electronic circuitry 612.

Support structure 600 attaches to support structure 608 to form theinsert casing. Internal surfaces of support structures 600 and 608 matetogether. Moreover, the internal surfaces of support structures 600 and608 can have cavities or extrusions to house and retain components ofthe sensing system. Externally, support structures 600 and 608 provideload bearing and articular surfaces that interface to the otherprosthetic components of the joint. The support structure 608 has asupport surface 610 that couples to a tibial implant. In general, thesupport surface 610 has a much greater load distributing surface areathat reduces the force, pressure, or load per unit area than thearticulating contact region of articular surfaces 602 and 604.

The support structures 600 and 608 can be temporarily or permanentlycoupled, attached, or fastened together. As shown, load sensing unit 170can be taken apart to separate support structures 600 and 608. A seal614 is peripherally located on an interior surface of support structure608. In one embodiment, the seal is an o-ring that comprises a compliantand compressible material. The seal 614 compresses and forms a sealagainst the interior surface of support structures 600 and 608 whenattached together. Support structures 600 and 608 form a housing wherebythe cavities or recesses within a boundary of seal 614 are isolated froman external environment. In one embodiment, a fastening element 620illustrates an attaching mechanism. Fastening element 620 has a lip thatcouples to a corresponding fastening element on support structure 600.Fastening element 620 can have a canted surface to motivate coupling.Support structures 600 and 608 are fastened together when seal 614 iscompressed sufficiently that the fastening elements interlock together.Support structures 600 and 608 are held together by fastening elementsunder force or pressure provided by seal 614 or other means such as aspring. Not shown are similar fastening elements that may be placed indifferent locations to secure support structures 600 and 608 equallyaround the perimeter if required.

In one embodiment, support structure 600 comprises material commonlyused for passive inserts. For example, ultra high molecular weightpolyethylene can be used. The material can be molded, formed, ormachined to provide the appropriate support and articular surfacethickness for a final insert. Alternatively, support structures 600 and608 can be made of metal, plastic, or polymer material of sufficientstrength for a trial application. In an intra-operative example, supportstructures 600 and 608 can be formed of polycarbonate. It should benoted that the long-term wear of the articular surfaces is a lesserissue for the short duration of the joint installation. The joint movessimilarly to a final insert when moved throughout the range of motionwith a polycarbonate articular surface. Support structure 600 can be aformed as a composite where a bearing material such as ultra highmolecular weight polyethylene is part of the composite material thatallows the sensing system to be used both intra-operatively and as afinal insert.

FIG. 7 illustrates one or more remote systems in an operating room inaccordance with an example embodiment. In general, an operating roomincludes a region where access is limited. Typically, personnel andequipment required for the procedure stay within a sterile field 702.Sterile field 702 is also known as a surgical field. Filtered air flowsthrough sterile field 702 to maintain the region free of contaminants.The airflow can be laminar through sterile field 702 to carrycontaminants away from the procedure thereby maintaining the sterileenvironment.

Remote systems as herein include a processor, communication circuitry,and a display. The communication circuitry couples to one or moredevices for receiving measurement data. The processor can process thedata to support a work-flow of the procedure. Processing of the data caninclude audio, visual, and haptic feedback in real-time that allowsrapid assessment of the information. In general, the measuredquantitative data can be displayed on a display that is in the line ofsight of the user but may not be in the sterile field. In a firstexample, a remote system 712 is attached to a ceiling. A mechanical armextends from the ceiling that allows remote system 712 to be moved to anappropriate height and angle for viewing within the sterile field.Remote system 712 includes a display 704 and a detector 706. In a secondexample, a remote system 714 is on a portable platform that can bewheeled to an appropriate location. Remote system 714 includes a display710 and a detector 706. One example of such a remote system is disclosedin U.S. patent application Ser. No. 12/723,486 entitled “SterileNetworked Interface for Medical Systems” filed Mar. 12, 2010 the entirecontents of which are hereby incorporated by reference.

Remote systems 712 and 714 operate in a similar fashion. The remotesystem 712 may be used for illustrative purposes but each example alsoapplies to portable remote system 714 and the corresponding componentsthereof. Although not shown, a tool, equipment, or device is used tomeasure a parameter of the muscular-skeletal system within surgicalfield 702. For example, a load sensing unit inserted in a prostheticknee joint wirelessly sends data to remote system 712. A navigationsystem can provide position and alignment information of the jointinstallation. A visualization of the prosthetic knee joint can bedisplayed on display 704 with the quantitative measurements related toload and alignment. The load sensing unit can provide quantitativemeasurements of the load magnitude and balance between condyles. Thenavigation system can provide information on alignment relative to themechanical axis of the leg. Furthermore, position data of the femur inrelation to the tibia can also be shown on remote system 712. Thequantitative measurement information displayed on remote system 712 canbe used select prosthetic components and effect changes before finalcomponents are installed.

The surgeon and other personnel within sterile field 702 can viewdisplay 704. The detector 706 of remote system 712 and similarlydetector 708 of remote system 714 includes multiple sensors that aredirected towards sterile field 702. Sensors included in detector 706 canbe but not limited to visual, audio, and motion sensors. The sensors canbe used to control the GUI on remote system remotely. For example,gestures can be received by detector 706 and identified that results inan action. Similarly, audible commands can be recognized that results inan action. The gestures can control the work-flow, how data ispresented, image size, a region of the screen, and access other programsto name but a few commands. Detector 706 provides a means of controllingremote system 712 from a distance. Detector 706 can be used to modify orchange a device used in a procedure. For example, device, equipment, ortool settings could be adjusted using gestures or audible commands thatmight otherwise be difficult during the procedure.

Detector 706 couples to the processor and display of remote system 712.Feedback can be provided by the system in response to the actionreceived by detector 706 from the user. The response can be visual,audio, haptic, or other means that is readily detected by the user. Theresponse can be provided by detector 706, display 704, or other devices.Similarly, visual, audio, or haptic feedback can be provided by detector706 or display 704 based on the quantitative data received fromequipment, instruments, or tools being used to assess themuscular-skeletal system of the patient for the prosthetic jointimplant. At this time, orthopedic surgeons rely on subjective feel withlittle or no quantitative measurement information related to combinedalignment, position, loading, and load balance. Remote systems 712 and714 are hands free operating room systems that allows the surgeon toreceive quantitative data in a non-obtrusive manner that supports anoptimal installation while reducing surgical time for the patient. Thequantitative data from each surgery can be data logged with thecorresponding work-flow step of the procedure for review and analysis asit relates to both short and long term outcomes.

FIG. 8 illustrates detector 706 detecting one of visual, motion, oraudio queue from a user 802. User 802 resides in sterile field 702 of anoperating room. Instruments, personnel, and the patient have beenremoved for clarity of the user operation. Detector 706 detects agesture 804 or motion used in the operation of the GUI of display 704.Motion detection or visual input from detector 706 is analyzed torecognize the command. In the example, user 802 wants to focus in on adifferent area of the screen. Gesture 804 is recognized and an action istaken. The example action as shown causes the screen to translate to thearea directed by user 802. In one embodiment, the system can providefeedback to the command recognition before the action is taken. Display704 or detector 706 can provide visual, audible, or haptic feedback toindicate recognition of the command. In some circumstances, remotesystem 712 may await a visual or audible confirmation to initiate theaction. Although shown with gestures, the command could be given audiblyor by the parameter measurement instrument used in the orthopedicprocedure coupled to

User 802 is providing a gesture 808 that moves display 704 back to theprevious screen. Detector 706 receives the visual or motion queue. Thevisual or motion queue is recognized resulting in an action to move backto the previous screen. In the example, remote system 704 can be coupledto the surgical navigation device and the load sensing unit.Quantitative measurement data on position, alignment, load, and loadbalance is provided on display 704. As mentioned previously, remotesystem 712 can be a computer system having one or more processors anddigital signal processors for analyzing the quantitative data and thevisual, motion, and audio queues from detector 706. Alternatively,detector 706 can have internal processors for analyzing detector inputfor recognition of commands and the resultant action. In general, asmall set of gestures that can control and allow interaction with theGUI on display 704 would result in better outcomes as quantitative datacan be added to subjective feel in a manner that adds little or no timeto how orthopedic surgeries are currently being performed.

User 802 is shown providing a gesture 812 that is detected by detector706. Detector 706 recognizes gesture 812 and responds by performing anaction corresponding to gesture 812. In the example, a step of thework-flow has been completed and gesture 812 results in the action thatthe remote system displays the next page of the work-flow. The new pageof the work-flow may require different parameter measurements to beperformed at different joint positions. The remote system 712 canrespond by visual, audible, haptic, or other means. In one embodiment,remote system 712 can respond to gesture 812 with an audible output 814.Audible output 814 can acknowledge the detection of gesture 812 and theaction taken thereof. Audible output 814 can further describe the actionbeing taken to ensure the correct action is being taken. Alternatively,the action can be indicated on screen 704. Audible output 814 can alsobe produced in response to measurement data output by the orthopedicmeasurement devices (not shown) coupled to remote system 712. Forexample, quantitative measured data related to alignment to themechanical axis can be analyzed by remote system 712. An audible output814 can be provided indicated that it is more than a predeterminedrange. Further audible output can be provided related to the amount ofvalgus or vargus misalignment. User 802 could continue work on thepatient getting real-time audio feedback on the measurements as itpertains to the steps of the work-flow. Similarly, audible output 814can be provided when the load sensing unit is placed in themuscular-skeletal system. The audible output 814 can recite loadmagnitude and load balance on the load sensing unit. Audible output 814can be non-verbal such as tone or sound magnitude changes for audiblyindicating measured parameters. Audible output 814 can be directed to anearpiece worn by user 802 as disclosed above. A further benefit can beachieved in providing audible direction of work-flow steps thatincorporates combined data from the navigation system and the loadsensing unit into a surgical related format that user 802 can perform onthe patient or prosthetic components.

User 802 can provide an audible command that is detected by detector706. Detector 706 or system 712 can have voice recognition software forassessing the audible command. Detector 706 upon recognizing audiblecommand 813 initiates an action as described herein. Audible commands818 can be used to control the GUI on display 704 or provide feedback inresponse to a visual or audible queue provided by remote system 712.Audible command 818 can also be used to change settings of instruments,equipment or tools coupled to remote system 712.

FIG. 9 illustrates a robotic tool 900 coupled to remote system 712 inaccordance with an example embodiment. In one embodiment, robotic tool900 includes an arm 914 for supporting operations such as bonemodifications and cuts for accepting prosthetic components. Robotic tool900, the load sensing unit, and the navigation system couple to remotesystem 712. Robotic tool 900 utilizes image data with actual points onthe patient muscular-skeletal system to map a region of interest. Arm914 of robotic tool 900 is motorized to precisely move in 3D space inrelation to the mapped region. Instruments and tools can be attached toarm 914. For example, a cutting device can be attached to arm 914 wherethe location in 3D space of the leading edge of the cutting device isknown and controlled by robotic tool 900. Robotic tool 900 can aid user802 in making precise and specific muscular-skeletal modifications. Inthe example, robotic tool 900 can be programmed to cut bone regions toreceive one or more prosthetic components. Robotic tool 900 can be usedin conjunction with the navigation system and the load sensing unit. Theuser interface on display 704 supports the work-flow that includesrobotic tool 900. In the example, quantitative data related to theposition of arm 914 of robotic tool 900, loading, joint position, andalignment can be provided on display 704. A gesture 902 can be used togenerate an action on robotic tool 900. Detector 706 detects andrecognizes gesture 902 and generates an action. Remote system 712 canprovide an audio output 910. In the example, audio output 910 can be aresponse to gesture 902. The audio output 910 can request confirmationto perform the action. User 802 can confirm by gesture 908 or audiblythat the action is correct and to initiate the action. An example of theaction is gesture 902 causing robotic arm 914 to move towards thesurgical area. Detector 706 detects and recognizes the confirmationgesture 908 and initiates robotic arm movement.

FIG. 10 illustrates a sensor array of detector 706 in accordance with anexample embodiment. Detector 706 has more than one sensor for visual,motion, and audio detection. In one example, detector 706 includes acamera 902 and one or more infrared sensors 904. Detector 706 can viewthe sterile field of the operating room scanning the space in 3D.Detector 706 can further include ultrasonic sensors 910. Ultrasonicsensors 910 can be used for motion detection within the sterile field. Asensor for distinguishing depth of image can further support 3Drecognition. Detector 706 further includes one or more audio transducers908. Transducers 908 can be a microphone for receiving audio and aspeaker for providing an audio output. As mentioned previously, detector706 can include one or more processors or digital signal processors forcapturing changes in image and motion. The one or more processors canutilize software for analyzing captured motion, voice recognition, soundgeneration, and voice generation. Recognition software can be used todetermine difference between normal physical activities within thesterile space from gestures. Similarly, the transducers 908 can be usedto identify and localized source vocalization such as the surgeon.Motion and sound recognition supports remote system 712 placed outsidethe sterile field for allowing user control of the GUI. Moreover, remotesystem 712 can reduce staff required in the operating room therebylowering exposure of contaminants to the patient. Remote system 712further facilitates the integration of measured data on themuscular-skeletal system that can quantify a subjective feel of asurgeon. The measured data can be provided verbatim in real-time ondisplay 704 of remote system 712. The quantitative measurements byitself may not be synthesized by the user rapidly nor may it be readilyadapted to the task being performed in the work-flow. Remote system 712can analyze the quantitative data and translate the measurements into anaction to be taken by the user that is a step of the work-flow. Forexample, load and load balance measurements can be translated can betranslated to specific soft tissue tensioning steps performed by thesurgeon to reduce load and increase balance. Thus, remote system 712 canprovide real-time feedback to measurement tools and the userrespectively based on measurements and motion/audio detection thatintegrates well with existing surgical work-flows.

The illustrations of embodiments described herein are intended toprovide a general understanding of the structure of various embodiments,and they are not intended to serve as a complete description of all theelements and features of apparatus and systems that might make use ofthe structures described herein. Many other embodiments will be apparentto those of skill in the art upon reviewing the above description. Otherembodiments may be utilized and derived therefrom, such that structuraland logical substitutions and changes may be made without departing fromthe scope of this disclosure. Figures are also merely representationaland may not be drawn to scale. Certain proportions thereof may beexaggerated, while others may be minimized. Accordingly, thespecification and drawings are to be regarded in an illustrative ratherthan a restrictive sense.

Such embodiments of the inventive subject matter may be referred toherein, individually and/or collectively, by the term “invention” merelyfor convenience and without intending to voluntarily limit the scope ofthis application to any single invention or inventive concept if morethan one is in fact disclosed. Thus, although specific embodiments havebeen illustrated and described herein, it should be appreciated that anyarrangement calculated to achieve the same purpose may be substitutedfor the specific embodiments shown. This disclosure is intended to coverany and all adaptations or variations of various embodiments.Combinations of the above embodiments, and other embodiments notspecifically described herein, will be apparent to those of skill in theart upon reviewing the above description.

These are but a few examples of embodiments and modifications that canbe applied to the present disclosure without departing from the scope ofthe claims stated below. Accordingly, the reader is directed to theclaims section for a fuller understanding of the breadth and scope ofthe present disclosure.

Where applicable, the present embodiments of the invention can berealized in hardware, software or a combination of hardware andsoftware. Any kind of computer system or other apparatus adapted forcarrying out the methods described herein are suitable. A typicalcombination of hardware and software can be a mobile communicationsdevice with a computer program that, when being loaded and executed, cancontrol the mobile communications device such that it carries out themethods described herein. Portions of the present method and system mayalso be embedded in a computer program product, which comprises all thefeatures enabling the implementation of the methods described herein andwhich when loaded in a computer system, is able to carry out thesemethods.

While the preferred embodiments of the invention have been illustratedand described, it will be clear that the embodiments of the inventionare not so limited. Numerous modifications, changes, variations,substitutions and equivalents will occur to those skilled in the artwithout departing from the spirit and scope of the present embodimentsof the invention as defined by the appended claims.

What is claimed, is:
 1. A medical system, comprising: a load sensingunit for measuring loading by converting a force applied to one or moreprosthetic components thereon within an anatomical joint into anelectric signal by way of piezoelectric transduction; a surgicalnavigation device communicatively coupled thereto for measuring positionand alignment of the prosthetic components and muscular-skeletal systemin relation to the loading and one another, where the surgicalnavigation system projects an ultrasonic sensing field and adjusts anenergy and range of the sensing field based on a proximity; and adisplay wirelessly coupled to the surgical navigation device and theload sensing unit for graphically presenting quantitative data on loadmagnitude, position, and alignment.
 2. The system of claim 1, furthercomprising an earpiece that audibly presents an indicator of thealignment and the loading responsive to one or more steps associatedwith identifying and tracking anatomical landmarks in the ultrasonicsensing field.
 3. The system of claim 1, wherein the surgical navigationdevice performs the steps of: emitting a plurality of ultrasonic pulsesfrom a plurality of ultrasonic transmitters on a wand of the surgicalnavigation device configured to transmit the ultrasonic pulses;capturing the ultrasonic pulses from a plurality of microphones on areceiver of the surgical navigation device; digitally sampling andstoring a history of received ultrasonic waveforms in a memory of thereceiver; estimating from the stored ultrasonic waveforms a time offlight between transmitting and receiving of the ultrasonic pulses;identifying a location of the wand from the time of flight measurementsreceived at the plurality of microphones; calculating for the pluralityof ultrasonic waveforms stored in the memory a phase differentialbetween the ultrasonic waveforms and previously received ultrasonicwaveforms; and updating the location of the wand from a mathematicalweighting of said time of flight with said phase differential.
 4. Thesystem of claim 3, wherein the receiver coordinates the emitting and thecapturing of the plurality of ultrasonic pulses via wireless infraredsynchronized communication from the wand.
 5. The system of claim 3,where the surgical navigation device comprises a wand and a receiverthat adjusts an ultrasonic waveform transmit sensitivity in accordancewith the proximity of the wand to the receiver.
 6. The system of claim3, where the surgical navigation device comprises a wand and a receiverthat adjusts an ultrasonic transmit sensitivity to compensate and matchtransmitter characteristics to airflow and temperature parameters of asound field therein.
 7. The system of claim 5, wherein the receivertransmits an index to the wand that identifies a waveshape stored inlocal memory and which the wand transmits in response to receiving theindex.
 8. An operating room system comprising: a load sensing unit forconverting an applied pressure associated with an internal load on ananatomical joint into an electrical signal thereby producing a loadmagnitude quantity; a surgical navigation device communicatively coupledthereto for creating an ultrasonic sensing field in air and withinproximity of the load sensing unit to report an alignment measureassociated with the load magnitude quantity; a remote system wirelesslycoupled to the surgical navigation device and the load sensing unit forreceiving and reporting the load magnitude quantity and alignmentmeasure; and a detector unit coupled to the remote system for receivingvisual, audio, or motion queues from within a surgical field forcontrolling the remote system.
 9. The system of claim 8, wherein thestep of converting the applied pressure associated with the force loadon the anatomical joint into an electric signal is performed by polymersensing, piezoelectric transduction, or ultrasonic waveguidedisplacement.
 10. The system of claim 8, where the remote systemcomprises a computer system coupled to a display, where the displaypresents load-line magnitude and alignment information as integratedquantitative data, and where the surgical navigation device adjusts astrength and range of the ultrasonic sensing field according to alocation of the surgical navigation device with respect to the loadsensing unit.
 11. The system of claim 9 where an action is generated bythe remote system in response to one of a visual, audio, or motionqueue.
 12. The system of claim 8, further comprising a displaywirelessly coupled to the surgical navigation device that graphicallypresents the load balance and alignment information.
 13. An operatingroom system for displaying a workflow that includes quantitativemeasurement data of the muscular-skeletal system comprising: one or moresensors for measuring a joint parameter of the muscular-skeletal system;a remote system coupled to the one or more sensors for displaying aworkflow of a procedure where quantitative measurements from the one ormore sensors related to the joint are displayed to support theprocedure; a detector unit coupled to the remote system for receivingvisual, audio, or motion queues from within a surgical field where anaction is generated upon receiving one of the visual, audio, or motionqueues.
 14. The system of claim 13, comprising a surgical navigationsystem for coupling to the remote system for providing position andalignment data of the muscular-skeletal system by measuring time offlight and differential time of flight ultrasonic waveforms.
 15. Thesystem 13, where the one or more sensors are load sensors and where theload sensors are within a load sensing unit for measuring load appliedby the muscular-skeletal system.
 16. The system of claim 13, where oneof the visual, audio, or motion queues controls a GUI of the remotesystem.
 17. The system of claim 13, where the detector unit includes acamera and at least one transducer.
 18. The system of claim 13, where anaction is performed by the remote system in response to the visual,audio, or motion queue received by the detector, and the action can bevisual, audible, or haptic.
 19. The system of claim 13, wherein one ofthe sensors converts an applied pressure associated with a force load onan anatomical joint into an electric signal by polymer sensing,piezoelectric transduction, or ultrasonic waveguide displacement. 20.The system of claim 13, comprising an earpiece that audibly presents anindicator of alignment and load balance responsive to receiving visual,audio, or motion queues from within a surgical field.